Detection of Utterances Directed to Robots by Situated Understanding in Object Manipulation Tasks

In this paper, we propose a novel method to detect robot-directed speech by situated understanding in human-robot physical interaction. The originality of this work is the introduction of a Multimodal Semantic Confidence measure based domain classification method, which is used to decide whether the speech can be interpreted as a feasible action under the current physical situation in an object manipulation task. This measure is calculated by integrating speech, image, and motion confidence with weightings that are optimized by logistic regression. Then we integrated this method with human attention, and conducted experiments under the conditions of natural human-robot interaction. keywords: robot-directed speech detection, multimodal semantic confidence, human-robot interaction